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CLAIMS 



Having thus described our invention, what we claim as new and desire to 
secure by Letters Patent is as follows: 



1 LA method for using future behavior of a battery of probes to forecast and 

2 diagnose network performance, comprising the steps of: 

3 consolidating normal network data and data returned from a battery of 

4 EPP probes into a single model matrix, a probe data set in said returned data 

5 including at least a time of said probe and a user performance measure; 

6 predicting a future value of said user performance measure from 

7 predictor variables in said single model matrix, values of said predictor 

8 variables being lagged from said probe time by a prediction horizon; 

9 generating an alarm signal if said predicted future value of the 

10 performance measure is above a threshold value; 

1 1 observing an actual value of said performance measure at the predicted 

12 time, said alarm signal being FALSE if the actual value is below a lower 

13 alarm threshold and an alarm signal being MISSED if no alarm signal was 

14 generated and the actual value is above an upper alarm threshold; and 

1 5 setting by trial and error said threshold value and said prediction 

16 horizon to achieve a rate of said false alarms below a specified limit. 

1 2. The method of claim 1, wherein said predicting step further comprises the 

2 steps of: 

3 reducing a dimensionality of said single model matrix by performing 

4 stepwise linear regression, at each step adding a most significant network 

5 node and deleting a least significant network node, a standard F test providing 

6 a significance statistic, until no addition or deletion is significant; 
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7 fitting functions of said predictor variables to an additive model; and 

8 generating from said additive model an estimated mean of said future 

9 value of said user performance measure of said EPP probe, an estimated 

10 variance of said estimated mean, and for each predictor variable an estimated 

1 1 variance of an estimated contribution to said future value of said predictor 

12 variable. 

1 3. The method of claim 1, further comprising the step of deriving a control 

2 scheme for said EPP probe by selecting coefficients for a formula for said 

3 alarm threshold, said alarm threshold formula including a constant term, a 

4 seasonality term and a term containing an estimated standard deviation of said 

5 future value of said user performance measure, wherein a rate of said missed 

6 alarms in said trial and error setting step is determined to be acceptable. 

1 4. The method of claim 1, wherein said model matrix includes for each 

2 EPP probe 

3 said user performance measure is a response time Y; 

4 said time of said probe is a time / when said response time Y was 

5 observed; 

6 said prediction horizon has a value h; and 

7 wherein for each of said predictor variables said lagged value is taken 

8 at, or most recently prior to, time t - h. 

1 5. The method of claim 1, further comprising the steps of: 

2 obtaining a list of models, one for each said linear regression step, a 

3 model for a last step providing a selected subset of predictor variables; 
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4 using a partial stepwise additive procedure to further reduce said 

5 dimensionality, said dimensionality being the number of a plurality of said 

6 network nodes; and 

7 determining a scale of EPP probe values, said scale being ordered by 

8 predictive power. 

1 6. The method of claim 1, further comprising the step of 

2 determining interaction terms for said additive model, said interaction terms 

3 being a function of more than one predictor variable. 

1 7. The method of claim 3, wherein said step of deriving a control scheme 

2 uses profile curves, said profile curves reflecting expected values of said user 

3 performance measure based on normally observed regularities in variation of 

4 values of said user performance measure. 



1 8. The method of claim 7, wherein said normally observed regularities 

2 include variations in a weekly cycle and variations in a daily cycle. 

1 9. The method of claim 1, further comprising the step of presenting a 

2 graphical summary of unfavorable predictors in response to triggering of said 

3 alarm threshold. 



1 10. The method of claim 2, further comprising the steps of 

2 determining whether said additive model is adequate and triggering a 

3 model alarm if said additive model is determined not to be adequate; and 

4 identifying elements of said additive model whose characteristics 

5 triggered said model alarm, said identifying step being initiated by said model 

6 alarm. 
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1 11. The method of claim 1 0, further comprising the steps of: 

2 selecting a new predictive model from a battery of recovery models; 

3 and 

4 building said new predictive model over time as data is accumulated 

5 for said predictor variables. 

1 12. The method of claim 1, wherein a predictive model is developed for a 

2 plurality of prediction horizons, one model at a time. 

1 13. A system for using future behavior of a battery of probes to forecast and 

2 diagnose network performance, comprising: 

3 a single model matrix comprising normal network data and responses 

4 to a battery of probes; 

5 profile curves corresponding to said battery, said profile curves 

6 reflecting expected response times based on recent history of said probes; 

7 an additive model that predicts the behavior of a function of said 

8 probes at a future time, said function being defined in terms of predictor 

9 variables in said model matrix, said function being selected by advanced 

10 statistical methods, and said additive model being applied cyclically to reduce 

1 1 a variable space of said predictor variables in said single model matrix; 

12 alarm thresholds based on deviation between said profile curves and 

13 behavior predicted by said additive model, said alarm thresholds being 

14 selected to assure a predictably low rate of false alarms; and 

1 5 a graphical summary of unfavorable predictors in response to 

16 triggering of said alarm threshold. 

1 14. A system as in claim 13, further comprising: 
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2 a monitor for determining whether said additive model is adequate, 

3 said monitor triggering a model alarm if said additive model is determined not 

4 to be adequate; and 

5 a diagnostic for identifying elements of said additive model whose 

6 characteristics triggered said model alarm, said diagnostic being initiated by 

7 said model alarm. 

1 15. A system as in claim 13, wherein said cyclic application of said model 

2 uses an F statistic to control addition and deletion of variables in said variable 

3 space. 

1 16. A system as in claim 13, wherein said cyclic application of said model 

2 uses a subset of variables determined by a stepwise linear regression (SLR). 

1 17. A computer implemented system for using future behavior of a battery of 

2 probes to forecast and diagnose network performance, comprising: 

3 computer code for consolidating normal network data and data 

4 returned from a battery of EPP probes into a single model matrix, a probe data 

5 set in said returned data including at least a time of said probe and a user 

6 performance measure; 

7 computer code for predicting a future value of said user performance 

8 measure from predictor variables in said single model matrix, values of said 

9 predictor variables being lagged from said probe time by a prediction horizon; 

10 computer code for generating an alarm signal if said predicted future 

1 1 value of the performance measure is above a threshold value; 

12 computer code for observing an actual value of said performance 

13 measure at the predicted time, said alarm signal being FALSE if the actual 

14 value is below a lower alarm threshold and an alarm signal being MISSED if 
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15 no alarm signal was generated and the actual value is above an upper alarm 

16 threshold; and 

17 computer code for setting by trial and error said threshold value and 

1 8 said prediction horizon to achieve a rate of said false alarms below a specified 

19 limit. 



1 18. The computer implemented system of claim 17, wherein said computer 

2 code for predicting further comprises: 

3 computer code for reducing a dimensionality of said single model 

4 matrix by performing stepwise linear regression, at each step adding a most 

5 significant network node and deleting a least significant network node, a 

6 standard F test providing a significance statistic, until no addition or deletion 

7 is significant; 

8 computer code for fitting functions of said predictor variables to an 

9 additive model; and 

10 computer code for generating from said additive model an estimated 

1 1 mean of said future value of said user performance measure of said EPP 

12 probe, an estimated variance of said estimated mean, and for each predictor 

13 variable an estimated variance of an estimated contribution to said future 

14 value of said predictor variable. 



1 19. The computer implemented system of claim 17, further comprising 

2 computer code for deriving a control scheme for said EPP probe by selecting 

3 coefficients for a formula for said alarm threshold, said alarm threshold 

4 formula including a constant term, a seasonality term and a term containing an 

5 estimated standard deviation of said future value of said user performance 

6 measure, wherein a rate of said missed alarms in said trial and error setting 

7 step is determined to be acceptable. 
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1 20. The computer implemented system of claim 18, further comprising: 

2 computer code for determining whether said additive model is 

3 adequate and for triggering a model alarm if said additive model is determined 

4 not to be adequate; and 

5 computer code for identifying elements of said additive model whose 

6 characteristics triggered said model alarm, said identifying step being initiated 

7 by said model alarm. 
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